Managing the Overall Balance of Operating System Threads on a Multiprocessor Using Automatic Self-Allocating Threads (ASAT)

A multi-threaded runtime environment which supports lightweight threads can be used to support many aspects of parallel processing including: virtual processors, concurrent objects, and compiler run-time environments. This work focuses on the area of compiler run-time environments. Such a library must depend on the underlying thread mechanism provided by the operating system. Threads working on compute intensive tasks work best when there is one thread performing real work on each processor. Matching the number of running threads to the number of processors can yield both good wall-clock run time and good overall machine utilization. The challenge is to schedule threads to maintain one running thread per processor by dynamically adjusting the number of threads as the load on the machine changes. It is generally not efficient to involve the operating system during a thread switch between lightweight threads. As such, a lightweight thread runtime environemnt must operate within the parameters provided by the operating system. This work identifies the situations on a multiprocessing system when the operation of a lightweight thread environment might be negatively impacted by other threads running on the system. The performance impact of these other threads is measured. A solution called Automatic Self Allocating Threads (ASAT) is proposed as a way to balance the number of active threads across a multiprocessing system. Our approach is significant in that it is designed for a system running multiple jobs, and it considers the load of all running jobs in its thread allocation. In addition, the overhead of ASAT is sufficiently small so that the run times of all jobs improve when it is in use. The improvements are achieved by eliminating contention for resources by jobs on the system. Furthermore, our approach uses self-scheduling so it is implemented in a run-time environment rather than in an operating system. Finally, the self-scheduling means that jobs need not all be scheduled by ASAT. This work is based on work supported by the National Science Foundation under Grant No MIP-0209402.